Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix file descriptor leaks in bindings and add test #11103

Merged
merged 1 commit into from
Aug 25, 2021

Conversation

jwhonce
Copy link
Member

@jwhonce jwhonce commented Aug 2, 2021

  • Add response.Body.Close() where needed to release HTTP
    connections to API server.
  • Add tests to ensure no general leaks occur. 100% coverage would
    required to ensure no leaks on any call.
  • Update code comments to be godoc correct

Signed-off-by: Jhon Honce [email protected]

@jwhonce jwhonce added kind/bug Categorizes issue or PR as related to a bug. HTTP API Bug is in RESTful API labels Aug 2, 2021
@jwhonce jwhonce requested review from baude and Luap99 August 2, 2021 21:58
@jwhonce jwhonce self-assigned this Aug 2, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 2, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jwhonce

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 2, 2021
@jwhonce jwhonce changed the title Fix file descriptor leaks and add test Fix file descriptor leaks in bindings and add test Aug 2, 2021
@jwhonce jwhonce force-pushed the wip/bindings branch 3 times, most recently from 03194e1 to 9f266de Compare August 2, 2021 22:44
@jwhonce jwhonce marked this pull request as draft August 2, 2021 23:11
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 2, 2021
@jwhonce
Copy link
Member Author

jwhonce commented Aug 3, 2021

@mheon https://github.com/containers/podman/pull/11103/files#diff-02a7031ca65fa34d69f3de533b25111aa07a03242c86c4660dbd35ec69d1cd2fR137 is causing me heartburn. Without a containers.Detach() it is not clear to me when the response.Body can be closed... /cc @baude

@Luap99
Copy link
Member

Luap99 commented Aug 3, 2021

@mheon https://github.com/containers/podman/pull/11103/files#diff-02a7031ca65fa34d69f3de533b25111aa07a03242c86c4660dbd35ec69d1cd2fR137 is causing me heartburn. Without a containers.Detach() it is not clear to me when the response.Body can be closed... /cc @baude

I think Attach() blocks until it reads EOF. So defer should work no?

@jwhonce
Copy link
Member Author

jwhonce commented Aug 3, 2021

@mheon https://github.com/containers/podman/pull/11103/files#diff-02a7031ca65fa34d69f3de533b25111aa07a03242c86c4660dbd35ec69d1cd2fR137 is causing me heartburn. Without a containers.Detach() it is not clear to me when the response.Body can be closed... /cc @baude

I think Attach() blocks until it reads EOF. So defer should work no?

@Luap99 I thought so, but the client fails to get any output when I defer the Close(). I have not had time to debug it yet.

@mheon
Copy link
Member

mheon commented Aug 3, 2021

@jwhonce Once Attach has handed over to Libpod, all responsibility for closing the connection is handed over to that code (the termination conditions are very complex, and the only place that has enough information to be able to know when they are met is Libpod). As such, that defer will never actually close the body, because it is guaranteed to already be closed by the end of the function.

Can you describe what's going on / why the body needs to be closed in more detail?

@jwhonce
Copy link
Member Author

jwhonce commented Aug 3, 2021

@mheon If we don't close body at the end of the operation we "leak" the file descriptor. From godoc

The client must close the response body when finished with it:

resp, err := http.Get("http://example.com/")
if err != nil {
	// handle error
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
// ...

@mheon
Copy link
Member

mheon commented Aug 4, 2021

@jwhonce I think the key is that we can only do that if the connection has not been hijacked (hijackChan in the handler indicates that this has happened). Once hijacking is complete, all responsibility for closing goes to the Libpod HTTPAttach function.

@jwhonce jwhonce force-pushed the wip/bindings branch 3 times, most recently from b30c8da to 9b95026 Compare August 9, 2021 17:07
@jwhonce jwhonce force-pushed the wip/bindings branch 2 times, most recently from 810f6eb to f16d52c Compare August 24, 2021 21:54
* Add response.Body.Close() where needed to release HTTP
  connections to API server.
* Add tests to ensure no general leaks occur. 100% coverage would be
  required to ensure no leaks on any call.
* Update code comments to be godoc correct

Signed-off-by: Jhon Honce <[email protected]>
@jwhonce jwhonce marked this pull request as ready for review August 25, 2021 16:50
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 25, 2021
Copy link
Member

@ashley-cui ashley-cui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rhatdan
Copy link
Member

rhatdan commented Aug 25, 2021

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 25, 2021
@openshift-merge-robot openshift-merge-robot merged commit 49cfed7 into containers:main Aug 25, 2021
@jwhonce jwhonce deleted the wip/bindings branch September 29, 2021 16:09
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. HTTP API Bug is in RESTful API kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants